On Search Engine Evaluation Metrics

نویسنده

  • Pavel Sirotkin
چکیده

2-3-Oh my God, a mistake!-It's not our mistake!-Isn't it? Whose is it?-Information Retrieval. " BRAZIL "-4-Acknowledgements One man deserves the credit, one man deserves the blame… TOM LEHRER, " LOBACHEVSKY " I would like to thank my supervisor, Wolfgang Stock, who provided me with patience, support and the occasional much-needed prod to my derrière. He gave me the possibility to write a part of this thesis as part of my research at the Department of Information Science at Düsseldorf University; and it was also him who arranged for undergraduate students to act as raters for the study described in this thesis. I would like to thank my co-supervisor, Wiebke Petersen, who bravely delved into a topic not directly connected to her research, and took the thesis on sightseeing tours in India and to winter beaches in Spain. Wiebke did not spare me any mathematical rod, and many a faulty formula has been spotted thanks to her. I would like to thank Dirk Lewandowski, in whose undergraduate seminar I first encountered the topic of web search evaluation, and who provided me with encouragement and education on the topic. I am also indebted to him for valuable comments on a draft of this thesis. I would like to thank the aforementioned undergraduates from the Department of Information Science for their time and effort on providing the data on which this thesis' practical part stands. Last, but definitely not least, I thank my wife Alexandra, to whom I am indebted for far more than I can express. She even tried to read my thesis, which just serves to show. As is the custom, I happily refer to the acknowledged all good that I have derived from their help, while offering to blame myself for any errors they might have induced.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Review of ranked-based and unranked-based metrics for determining the effectiveness of search engines

Purpose: Traditionally, there have many metrics for evaluating the search engine, nevertheless various researchers’ proposed new metrics in recent years. Aware of this new metrics is essential to conduct research on evaluation of the search engine field. So, the purpose of this study was to provide an analysis of important and new metrics for evaluating the search engines. Methodology: This is ...

متن کامل

A Survey on Performance Evaluation Measures for Information Retrieval System

information to the users. To make the search effective, a tool called search engine has been introduced. These engines crawl the web for the given users query and display the results to the user based on the relevance score (ranking). Different search engine employs different ranking algorithm. Many ranking algorithm is being introduced frequently by several researchers. Several metrics are ava...

متن کامل

Performance Evaluation of search engines via user efforts measures

Many metrics exist to perform the task of search engine evaluation that are either looking for the experts judgments or believe in searchers decisions about the relevancy of the web documents. However, search logs can provide us information about how real users search. This paper explains, our attempts to incorporate the users searching behavior in formulation of user efforts centric evaluation...

متن کامل

Evaluating Web Search Result Summaries

The aim of our research is to produce and assess short summaries to aid users’ relevance judgements, for example for a search engine result page. In this paper we present our new metric for measuring summary quality based on representativeness and judgeability, and compare the summary quality of our system to that of Google. We discuss the basis for constructing our evaluation methodology in co...

متن کامل

Employing log metrics to evaluate search behaviour and success: case study BBC search engine

This paper argues that metrics can be generated from search transactional web logs that can help evaluate search engine effectiveness. Search logs from the BBC website were analysed and metrics extracted. Two search metrics – the time lapse between searches and the number of searches in a session – were developed to see whether they could measure search success or satisfaction. In all, 4 millio...

متن کامل

Predicting User Preferences

The many metrics employed for the evaluation of search engine results have not themselves been conclusively evaluated. We propose a new measure for a metric’s ability to identify user preference of result lists. Using this measure, we evaluate the metrics Discounted Cumulated Gain, Mean Average Precision and classical precision, finding that the former performs best. We also show that consideri...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • CoRR

دوره abs/1302.2318  شماره 

صفحات  -

تاریخ انتشار 2013